We revisit the classical analysis of generative vs discriminative models for general exponential families, and high-dimensional settings. Towards this, we develop novel technical machinery, including a notion of separability of general loss functions, which allow us to provide a general framework to obtain l convergence rates for general M-estimators. We use this machinery to analyze l and l2 convergence rates of generative and discriminative models, and provide insights into their nuanced behaviors in high-dimensions. Our results are also applicable to differential parameter estimation, where the quantity of interest is the difference between generative model parameters.

This paper starts by developing a notion of local "separability" of a loss function, which they use to get l_infty convertence rates, in terms of the separability parameters, for low and high dimensional settings. These rates are then applied to then applied to a probabilistic classification problem with both a generative and discriminative approach. After computing the teh separability parameters for each, they can apply the theorems to get l_infty convergence rates for the discriminative approach (logistic regression), as well as two generative approaches (for the cases that x y is isotropic Gaussian and gaussian graphical model). They next consider l_2 convergence rates. The discriminative rate is trivial based on the support consistency and the l_infty rates.

We revisit the classical analysis of generative vs discriminative models for general exponential families, and high-dimensional settings. Towards this, we develop novel technical machinery, including a notion of separability of general loss functions, which allow us to provide a general framework to obtain l convergence rates for general M-estimators. We use this machinery to analyze l and l2 convergence rates of generative and discriminative models, and provide insights into their nuanced behaviors in high-dimensions. Our results are also applicable to differential parameter estimation, where the quantity of interest is the difference between generative model parameters. Papers published at the Neural Information Processing Systems Conference.